Elon on Mars
The Bear in the Pool
Painting on the Wall
This project aims to implement methods to insert an image into another one seamlessly.
The objective of this problem is to produce an output image that resembles the source image by solving a least squares problem. Matrix A should contain the following constraints: differences between both x and y gradients of the output image and those of the source should be close, and the pixel intensity at coordinate (0,0) in the output image should be close to that of the source image. Since each constraint contains only two non-zero variables, I used a sparse matrix least-squares solver.
This method calculates output intensities v using the intensities of both the source image s and the target image t. This is a least squares problem to solve the following objective function.
Matrix A contains the following constraints: differences between output intensities should be close to those of the source within the source region, and differences between output intensities and target intensities should be close to the differences between those of the source within the boundaries. I used a sparse matrix least-squares solver. I solved the optimization problem for each RGB channel separately.
For Bells & Whistles, I implemented the followings:
I used the Mixed Gradient method which is like Poisson blending except it uses the maximum gradient of source and target pixels.
The Mixed Gradient blending improves the quality and seamlessness of results where the foreground image needs to have the same texture or color as the background. For example, if the item in the foreground image is transparent or it needs to be incorporated in the background. In one of my examples, I tried to add a picture of Brad Pit as a wall painting on a brick wall. By comparing the results of the Mixed Gradient and Poisson blending, we can see how the incorporation of wall texture in the foreground image makes the image more natural for a wall painting.
My favorite result is the first one.
| Naive Blend | Poisson Blend | Mixed Blend |
|---|---|---|
|
|
|
|
Acceptable The result seems acceptable. However, as a painting on a brick wall, we expect to see the wall's texture through the foreground image. |
Success Using the mixed gradient blend has incorporated the wall's texture in the foreground image and makes more like painint on a brick wall. |
|
|
|
|
|
Failure This is an intersting result. The difference of gradients between background image and foreground image is very high. So, the modifications in the foreground image are huge which makes the image too bright. |
Failure The foreground image is too bright and also transparent. |
|
|
|
|
|
Success The foreground image has became very red. However, the result is good enough (it's on the Mars after all). |
Failure The foreground image is transparent. |
|
|
|
|
|
Success The result seems fine and seamless enough. |
Failure Again, the foreground image has become transparent. |
|
|
|
|
|
Failure The belending result seems unnatural. we expect (logically) that the person on the moon should have a brightness like moon. |
Failure The result is not acceptable. |
|
|
|
|
|
Success The results is very seamless and natural. |
Acceptable At first glance, the result seems good. However, if we pay attention to the bear's head, we can see the waves pattern in the bear's head. This makes it a little unnatural. |
By converting the RGB image to grayscale, the contrast between the colors would be lost. To solve the issue, I converted the RGB image to an HSV image and used the Mixed Gradient method on grayscale image as background and each of H, S, and V channels as foreground image. Based on the results, this method works best with V channel.
The results of H channel seem acceptable as well. However, the results look unnaturally darker. In some cases, the circles in the blend are darker than the grayscale results. That's why I think the method works best with Channel V.
| Original Image | CV2 Color2Gray | Mixed Channel H | Mixed Channel S | Mixed Channel V |
|---|---|---|---|---|
|
|
|
|
|
|
Failure The number 3 is not visible. |
Success The number 3 is visible. |
Failure The number 3 is slightly visible. |
Success The number 3 is visible. |
|
|
|
|
|
|
|
Failure The number 7 is not visible. |
Acceptable Although the number 7 is visible, the colors are darker. |
Failure The number 7 is slightly visible. |
Success The number 7 is visible. |
|
|
|
|
|
|
|
Failure The number 8 is not visible. |
Success The number 8 is visible. |
Failure The number 8 is slightly visible. |
Success The number 8 is visible. |
|
|
|
|
|
|
|
Failure The number 8 is not visible. |
Acceptable Although the number 8 is visible, the colors are too dark. |
Failure Although the number 8 is visible, the colors are unnaturally dark or bright. |
Success The number 8 is visible. |